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ABSTRACT 



A data receiving apparatus is formed of a set top unit 
connected to a network and a security module. Digital video 
data which is supplied from the network and scrambled 
according to a first system is scrambled according to a 
second system in a scramble circuit in the set top unit, and 
is supplied to the security module. The data is descrambled 
according to the first system in a descramble circuit in the 
security module, and is transferred back to the set top unit. 
The data is descrambled according to the second system in 
a descramble circuit in the set top unit, and is outputted to 
an image display terminal via an MPEG decoder. 

39 Claims, 26 Drawing Sheets 
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APPARATUS AND METHOD FOR 
RECEIVING AND DECRYPTING 
ENCRYPTED DATA AND PROTECTING 
DECRYPTED DATA FROM ILLEGAL USE 

BACKGROUND OF THE INVENTION 

The present invention relates to a data receiving apparatus 
which reproduces and outputs multimedia data such as 
images and characters transmitted from, for example, a 
computer network and a digital broadcasting network, or 
multimedia data such as images and characters outputted 
from drives of information media such as CD-ROM and 
DVD to a personal computer and a television receiver, and 
more specifically, relates to a data receiving apparatus which 
has a function to protect decrypted data transmitted or 
outputted with being encrypted. 

This application is based on Japanese Patent Application 
No. 8-233100, filed Sep. 3, 1996, No. 8-300407, filed Nov. 
12, 1996, No. 9-112539, filed Apr. 30, 1997, and No. 
9-112540, filed Apr. 30, 1997, the content of which is 
incorporated herein by reference. 

In recent years, there have been proposed a data receiving 
apparatus which receives so-called multimedia data trans- 
mitted from digital broadcasting networks such as a com- 
puter network and digital satellite broadcasting, and from 
data storage media such as DVD (Digital Versatile Disc), 
and which outputs them into a personal computer and a 
television receiver. When each home is equipped with such 
a data receiving apparatus, a service providing system of 
multimedia data including a video, sound and characters 
such as video-on-demand service can be realized. 

Consequently, where the above-mentioned service pro- 
viding system is constructed, it is important to protect rights 
such as a copyright of creators and data providers 
(individual and companies) of multimedia data. Since mul- 
timedia data are provided as digital data, it is easy to copy 
the data by using a personal computer. Moreover, where 
multimedia data are converted into an analog signal (video 
signal, etc.) so as to be supplied to a television receiver, it is 
easy to record the data into a video tape by a video cassette 
recorder. Therefore, where a system which distributes mul- 
timedia data efficiently is constructed, the problem of pro- 
tecting rights of data providers is inevitable and important 
matter. 

In general, in order to prevent a third party who is not a 
genuine contractor from receiving data without fee, a service 
provider provides encrypted data. The service provider 
informs only the contractors of encryption keys so that only 
the contractors can decrypt the encrypted data correctly. 
Here, data are supplied to the contractors through not only 
a cable network but also a wireless LAN, television 
broadcasting, a storage medium, etc. 

As a conventional example of such a data receiving 
apparatus, there is an apparatus shown in FIG. 1. This 
apparatus comprises a set top unit 10, a security module 20 
and an IC card 30, but the set top unit 10 and the security 
module 20 form one unit actually, and thus one product 
(information receiving apparatus) is realized. Therefore, 
only IC card 30 is separated from this product. 

In the case of a network, wireless LAN or television 
broadcasting, encrypted data (here, encrypt means scramble, 
and hereinafter, encrypted data is referred to as scrambled 
data) supplied from an antenna or outputted from a storage 
medium are inputted into a receiver/demodulator 12 of the 
set top unit 10. In this example, original data is digital image 
data which have been encoded by the MPEG encoding 



;i,45i 

2 

algorithm. Output of the receiver/demodulator 12 
(scrambled data) is supplied to the security module 20, and 
is inputted to a descramble circuit 22 and a filter 24. 
The filter 24 extracts ECM data and EMM data from 

5 inputted stream data, and supplies them to an interface 26. 
The ECM and EMM data are data which are defined in the 
MPEG-2, and ECM means Entitlement Control Message, 
and EMM means Entitlement Management Message. 
Concretely, they are a stream including control data which 

10 flows in an MPEG transport stream when payload of a 
packet of the MPEG transport stream which transports an 
image and sound data is scrambled. 

The ECM includes data for access control which is 
peculiar to the streams of image and sound data such as a key 

15 necessary for releasing the scramble, a program number (in 
the MPEG-2, the program means a group of image and 
sound stream data having a common time base), and the fee 
for the program. 

The EMM includes data for access control relating to the 
whole system. For example, the EMM includes names of 
new users and new program numbers. 

In such a manner, the filter 24 performs filtering for 
extracting the streams (in the case of a program stream, a 

25 program stream map, and in the case of a transport stream, 
a TS program map section) including ECM and EMM 
transmitted mixing with the streams of the image and sound 
data encoded by the MPEG encoding algorithm. Such fil- 
tering is performed according to PIDs (packet ID) allocated 

30 to the packets and values of stream IDs. 

The interface 26 is connected to the IC card 30 owned by 
a contractor. The service provider who transmits digital 
image data previously writes a descramble key correspond- 
ing to scramble of transmission, a password of an user, etc. 

35 into the IC card 30, and hands it to the user upon the 
contraction. 

In the system shown in FIG. 1 owning the data receiving 
apparatus (formed of the set top unit 10 and the security 
module 20) means a sort of authentication, but in order to 

40 cope with an apparatus theft, authentication such as pass- 
word verification is actually performed. 

For this reason, the IC card 30 is connected to the 
interface 26, the authentication is performed by using a 
user's password which has been previously written into the 

45 IC card 30. When the authentication is finished successfully, 
the descramble key or secret data required for decrypting 
encrypted descramble key is inputted from the IC card 30 
into the descramble circuit 22 in the security module 20. 

50 The descramble circuit 22 descrambles scrambled data 
supplied from the set top unit 10 by using the descramble 
key, and transmits the original digital image data coded by 
the MPEG method back to the set top unit 10. The original 
image data are outputted through a demultiplexer 14 and an 

55 MPEG decoder 16 in the set top unit 10 into a user terminal 
(image display unit, etc.), not shown. The MPEG decoder 16 
includes an analog/digital converter, and outputs an analog 
video signal corresponding to the original image data. 
In such a manner, the scrambled data are descrambled 

60 according to the security module 20, and the original digital 
image data coded by the MPEG method are supplied to the 
set top unit 10. Therefore, only a genuine user having the 
descramble key in the IC card 30 can descramble the 
scrambled data. 

65 However, in the data receiving apparatus, the user's secret 
information such as the descramble key and the password 
appears in the interface 26. Therefore, the user's secret 
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information can be stolen by a third party through the 
interface 26, and thus there arises problems of the user 
protection and security. 

In order to avoid such a problem, there is proposed that 
the IC card 30 and the security module 20 form one unit (the s 
set top unit 10 is separated from the security module 20). In 
this case, user's secret information is not stolen by a third 
party, but since original digital image data, which have been 
descrambled, appears at the interface between the security 
module 20 and the set top unit 10, the digital data can be 10 
utilized (copied, etc.) illegally, and this threatens the service 
provider. 

The above-mentioned problems are not limited to the case 
of receiving and decrypting data supplied through a 
network, so the problems arise similarly in the case of 1 
distribution of packaged software etc. 

As described above, the conventional data receiving appa- 
ratus was insufficient to protect encrypted data. 

20 

BRIEF SUMMARY OF THE INVENTION 

Accordingly, it is an object of the present invention to 
provide a data receiving apparatus which can prevent illegal 
use of encrypted data. 

According to the present invention, the data receiving 25 
apparatus comprising first and second units. The first unit 
encrypts data, which have been encrypted according to a first 
encrypt system and supplied, according to a second encrypt 
system, and transmits the encrypted data to the second unit. 
The second unit decrypts the encrypted data according to a 30 
first decrypt system, and transfers the data back to the first 
unit. 

Therefore, since only the data which have been encrypted 
according to the second encrypt system appears in an 35 
interface between the first and second units, illegal use of the 
decrypted data is prevented. Moreover, since user's secrete 
data are not outputted from the second unit to outside, the 
user's secret data can be protected securely. 

Additional objects and advantages of the present inven- 40 
tion will be set forth in the description which follows, and 
in part will be obvious from the description, or may be 
learned by practice of the present invention. 

The objects and advantages of the present invention may 
be realized and obtained by means of the instrumentalities 45 
and combinations particularly pointed out in the appended 
claims. 

BRIEF DESCRIPTION OF THE SEVERAL 

VIEWS OF THE DRAWING 50 

The accompanying drawings, which are incorporated in 
and constitute a part of the specification, illustrate presently 
preferred embodiments of the present invention and, 
together with the general description given above and the 
detailed description of the preferred embodiments given 55 
below, serve to explain the principles of the present inven- 
tion in which: 

FIG. 1 is a block diagram showing a data receiving 
apparatus according to a conventional example; 6Q 

FIG. 2 is a block diagram showing a data receiving 
apparatus according to a first embodiment of the present 
invention; 

FIG. 3 is a schematic drawing explaining a function of the 
first embodiment; 65 

FIG. 4 is a block diagram showing details of the key 
control circuit of FIG. 2; 
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FIG. 5 is a block diagram showing a whole system 
including the data receiving apparatus according to a second 
embodiment of the present invention; 

FIG. 6 is a block diagram showing the data receiving 
apparatus according to the second embodiment; 

FIG. 7 is a block diagram showing an IC card according 
to the second embodiment; 

FIG. 8 is a flow chart of the second embodiment; 
FIG. 9 is a block diagram showing a fingerprint circuit of 
the second embodiment; 

FIG. 10 is a block diagram showing the data receiving 
apparatus according to a third embodiment of the present 
invention; 

FIG. 11 is a block diagram showing the data receiving 
apparatus according to a fifth embodiment of the present 
invention; 

FIG. 12 is a block diagram showing the data receiving 
apparatus according to a sixth embodiment of the present 
invention; 

FIG. 13 is a block diagram showing the data receiving 
apparatus according to a seventh embodiment of the present 
invention; 

FIG. 14 is a block diagram showing the data receiving 
apparatus according to an eighth embodiment of the present 
invention; 

FIG. 15 is a block diagram showing the data receiving 
apparatus according to a ninth embodiment of the present 
invention; 

FIG. 16 is a block diagram showing the data receiving 
apparatus according to a tenth embodiment of the present 
invention; 

FIG. 17 is a block diagram showing the data receiving 
apparatus according to an eleventh embodiment of the 
present invention; 

FIG. 18 is a block diagram of a fingerprint circuit accord- 
ing to the eleventh embodiment; 

FIG. 19 is a flow chart showing a function according to 
the eleventh embodiment; 

FIG. 20 is a block diagram showing the data receiving 
apparatus according to a twelfth embodiment of the present 
invention; 

FIG. 21 is a block diagram showing the data receiving 
apparatus according to a thirteenth embodiment of the 
present invention; 

FIG. 22 is a block diagram of a data separating circuit and 
a fingerprint circuit according to the thirteenth embodiment; 

FIG. 23 is a block diagram of the data separating circuit 
according to the thirteenth embodiment; 

FIG. 24 is a flow chart showing a function of the thirteenth 
embodiment; 

FIG. 25 is a block diagram of a main section of the data 
receiving apparatus according to a fourteenth embodiment 
of the present invention; 

FIG. 26 is a block diagram of a main section of the data 
receiving apparatus according to a fifteenth embodiment of 
the present invention; 

FIG. 27 is a flow chart showing a function of the fifteenth 
embodiment; 

FIG. 28 is a block diagram showing a modification of the 
fifteenth embodiment; and 

FIG. 29 is a block diagram showing the data receiving 
apparatus according to a sixteenth embodiment of the 
present invention. 
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DETAILED DESCRIPTION OF THE 
INVENTION 

A preferred embodiment of a data receiving apparatus 
according to the present invention will now be described 
with reference to the accompanying drawings. 

First Embodiment 

FIG. 2 is a block diagram of the first embodiment. The 
present embodiment comprises a set top unit 50 and a 
security module 70. They are different from the conventional 
example shown in FIG. 1, and are separated from each other. 
They are detachable and an interface exists therebetween. 

The set top unit 50 comprises a receiver/demodulator 52, 
a scramble circuit 54, a descramble circuit 56, a demulti- 
plexer 58, an MPEG decoder 60 and a key control circuit 62. 
The security module 70 comprises a descramble circuit 72 
and an authentication/access controller 74. Here, the security 
module 70 may be realized not as a unit but an IC card. 

Similarly to the conventional example, encrypted data 
(scrambled MPEG digital image data) supplied from a 
network or an antenna are inputted into the receiver/ 
demodulator 52 of the set top unit 10. The data is scrambled 
according to a server of data (not shown). This scramble 
process is referred to as the first scramble process S A . An 
output of the receiver/demodulator 52 is supplied to the 
scramble circuit 54 which performs the second scramble 
process S fl different from the first scramble process S A and 
to the key control circuit 62 which controls a key of the 
second scramble process. 

When data is supplied from the receiver/demodulator 52 
to the key control circuit 62, the key control circuit 62 
generates the scramble key for the second scramble process 
S A and corresponding descramble key, and supplies the 
scramble key and descramble key respectively to the 
scramble circuit 54 and the descramble circuit 56. Here, 
unlike the conventional example shown in FIG. 1, since 
those keys are not outputted outside the set top unit 50, 
user's secret data are protected securely. When the first and 
second descramble processes are respectively D A and D B , 
the key control circuit 62 generates a scramble key for the 
second scramble process S A and a descramble key so that 
D fl -D A -S^ S A =I (I: identity transformation) is satisfied. 

The scramble circuit 54 performs the second scramble 
process S B by using the scramble key supplied from the key 
control circuit 62. The output of the scramble circuit 54 is 
supplied to the security module 70, and is inputted into the 
descramble circuit 72 for performing the first descramble 
process D A , 

The descramble circuit 72 performs the first descramble 
process D A on the data supplied from the set top unit 50 
using a descramble key supplied from the authentication/ 
access controller 74, and transmits the descrambled data 
back to the set top unit 50. The service provider who 
transmits the digital image data previously writes a 
descramble key corresponding to the first scramble process 
S A upon transmission into the authentication/access control- 
ler 74, and hands it to a user upon contraction. Therefore, 
data in which the first scramble is released from the data 
supplied from the network or antenna to the set top unit 50 
are obtained from the descramble circuit 72. However, the 
second scramble process S A has been performed on the data 
by the scramble circuit 54. 

Into the authentication/access controller 74 instead of the 
conventional IC card, a descramble key, a user's password, 
etc. are written by the service provider, and owning of the 
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security module 70 including the authentication/access con- 
troller 74 means a kind of authentication. 

In the set top unit 50, the descramble circuit 56 performs 
the second descramble process D^ on the input data using a 

5 descramble key supplied from the key control circuit 62, and 
reproduces the original MPEG encoded digital image data. 
The output of the descramble circuit 56 are outputted into a 
user's terminal (image display unit, etc.), not shown, via the 
demultiplexer 58 and the MPEG decoder 60. The MPEG 

10 decoder 60 includes an analog/digital converter, and outputs 
an analog video signal. 

The following describes an operation of the present 
embodiment with reference to FIG. 3. FIG. 3 shows only the 
scramble process and descramble process, and here, shows 

35 also a first scramble circuit 42 for performing the first 
scramble process S A in a server 40 of a service provider. 
When the original digital data is M, the first scramble circuit 
42 in the server 40 outputs data S A (M) which have been 
scrambled according to the first method. 

20 When this data is received by the set top unit 50, the 
second scramble circuit 54 performs the second scramble 
process S B on the data, and outputs data S fi (S A (M)). 
Therefore, data, which have been duplex-scrambled accord- 
ing to the first and second scramble methods, are supplied 

25 from the set top unit 50 to the security module 70. Since 
even if the data are stolen by a third party, the data cannot 
be descrambled, the original data cannot be reproduced, and 
thus the original digital data cannot be utilized illegally. 

30 The first descramble circuit 72 in the security module 70 
performs the first descramble process D A on the duplex- 
scrambled data, and outputs data D A (S^(S A (M)))=S £ (M), 
and transmits the data S B (M) back to the set top unit 50. 
Therefore, data scrambled according to the second method 

35 are supplied from the security module 70 to the set top unit 
50. Since even if the data are stolen by a third party, the data 
cannot be descrambled, the original data cannot be 
reproduced, and thus the original digital data cannot be 
utilized illegally. In particular, since the key for the second 

40 scramble process is generated by the key control circuit 
62 in the set top unit 50, it does not leak outside, and illegal 
utilization of the original data by a third party can be 
prevented. 

The second descramble circuit 56 in the set top unit 50 

45 performs the second descramble process D B on the input 
data, and outputs data D^(D A (sj(S A (M)))). As described 
above, since the key control circuit 62 selects the second 
scramble/descramble key so that the second scramble 
process/descramble process S B and satisfy 

50 D £ D A S^ S A =I, the output data D*(D A (S^S A (M))))=M, and 
thus the descramble circuit 56 can reproduce the original 
data M. Here, the condition of D F D A S fl S A =I does not 
require the condition of D^S^-D^-S^I. 
As described above, according to the present 

55 embodiment, since the original digital data do not appear in 
the interface between the set top unit 50 and the security 
module 70, the original data cannot be utilized (copied, etc.) 
illegally, and thus the service provider can be protected 
sufficiently. Further, since the interface does not exist 

60 between the IC card and the security module unlike the 
conventional example, the user's secrete data such as a 
password and a scramble key are not stolen by a third party. 

Further, in order to improve the security, it is more 
efficient that the key control circuit 62 changes the scramble/ 

65 descramble keys for the second method regularly/ 
irregularly. Namely, by monitoring and cryptoanalyzing the 
data outputted from the set top unit 50, there is a few 
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possibility of detecting the scramble/descramble keys for the insert, digital watermarking, and digital fingerprint. In each 

second method. However, by making the keys variable and data processing technique, basically, specific data 

by updating the keys frequently, such a possibility can be (characters, a figure, a mark, etc.) representing a copyright, 

substantially zero. for example, are concealedly inserted into digital image data 

FIG. 4 is a detailed circuit diagram of the key control 5 such as an image to be protected. Even if the original digital 

circuit 62. The key control circuit 62 comprises a key image data are normally reproduced, the specific data cannot 

generator 62b for generating scramble/descramble keys and be confirmed with them being concealed, but they can be 

a controller 62a for controlling a key generation and a key reproduced by a specific reproducing process, 

update and for interfacing with other circuits. When the data provider provides digital image data such 

The key generator 62b generates a new key in response to 10 as an image to a user, the data provider can insert the specific 

a key generation control signal from the controller 62a and data representing a copyright into the digital image data, etc. 

supplies back the new key to the controller 62a. The key by using such a data processing technique without notifying 

generator 62b includes a random number generator for the user (in other words, without deteriorating the original 

generating a key. The controller 62a supplies parameters image data, etc.). However, only when the specific data 

necessary for generating the random number, such as a seed 15 representing a copyright are inserted, the provided data are 

for random number generation, the length of the key, and a copied into a medium such as an optical disk, and thus there 

restriction for the key. Alternately, the key generator 62b can are still problems as to the actual effect in preventing illegal 

have a function to obtain such parameters by itself, distribution. 

The controller 62a instructs the key generator 62b to Therefore, there considers a method of inserting identi- 

obtain the new keys and supplies the new keys to the 20 fication (ID) data for identifying a user who is provided with 

scramble circuit 54 and the descramble circuit 56 to instruct data as the specific data. In this method, where data are 

them an update of key. If necessary, the controller 62a also copied illegally to be distributed, it becomes easy to specify 

supplies to the scramble circuit 54 and the descramble a user who copied the data by reproducing its ID data. As a 

circuit 56 an information about a position in the stream data result, the provider can bring an action against the user who 

where a key is updated in the scramble circuit 54 and the 25 copied the data illegally and distributed them, thereby mak- 

descramble circuit 56. ing il possible to effectively realize the protecting of nghts 

Since the set top unit 50 is separated from the security of thc data provider, 

module 70, the following effects are produced. The set top However, where as the data distributing system, particu- 

unit 50 can be commonly used by a plurality of users. larly a system of transmitting data to many and unspecified 

Namely, only one set top unit 50 is arranged in a home, and users such as a computer network and a digital broadcasting 

each member of the family can own the peculiar security network is assumed, it is practically difficult to insert ID data 

module 70. Moreover, it is considered that the scramble per user. 

method varies with service providers, but in this case, by EmbodimeQl 

incorporating the descramble functions peculiar to the ser- 35 

vice providers into the security module, a user can cope with Therefore, as the second embodiment, it will be described 

this case by using one set top unit 50. an embodiment which can easily insert ID data into provided 

Here, the above explanation refers to the scramble as the data per user even if the data service system of transmitting 

encryption, but the encryption is not limited to this, the data to many and unspecified users is used, and, as a result, 

encryption may be normal methods such as the RSA method 4Q which can securely protect rights such as a copyright of a 

and the DES method. Moreover, data supplied from a data provider. 

network are not limited to image data, they may be sound FIG. 5 is a block diagram showing a constitution of the 

data, video data, etc. Further, the supply of data is applied whole data communication system according to the present 

not only to the case of the supply via a network but also to embodiment, FIG. 6 is a block diagram showing a main 

the case of the supply via a storage medium. 45 section of the data receiving apparatus relating to the second 

In the above embodiment, the descramble key is stored in embodiment, and FIG. 7 is a block diagram showing a 

the security module 70. However, it is possible to transmit constitution of the IC card used together with the data 

the descramble key from the network to the security module receiving apparatus according to the second embodiment. 

70 in an encrypted manner. When a hierarchical encryption According to the present embodiment, as shown in FIG. 

is employed, the present invention can be applied if the 50 5, the data service system in which a data transmitting 

decryption key of the uppermost layer of the hierarchical apparatus 103 is connected to the data receiving apparatus 

encryption is stored in the security module 70. 101 via a network 102 is assumed. The network 102 may be 

Other embodiments of the data receiving apparatus a digital broadcasting network for a computer network 
according to the present invention will be described. The (internet) and digital satellite broadcasting. The data trans- 
same portions as those of the first embodiment will be 55 mitting apparatus 103 is assumed as an apparatus in the side 
indicated in the same reference numerals and their detailed of data service provider who provides various data (so- 
description will be omitted. called multimedia data) including an image. 

In the first embodiment, since the original digital data do The data receiving apparatus 101 is a receive only appa- 

not appear in the interface between the set top unit 50 and ratus set per user such as a home, and it is a main element 

the security module 70, the illegal utilization (copying, etc.) <so of a data reproducing apparatus including the IC card 110, 

of the original digital data can be prevented, but recording a monitor 104 and a local media 105, described later. The 

an analog video signal supplied to a user's terminal into a monitor 104 is, for example, a personal computer or a 

video tape by a video cassette recorder, etc. cannot be television receiver having a display device and if necessary, 

prevented. other reproducing devices such as a loudspeaker. The local 

Meanwhile, as a technique for realizing the protection of 65 media 105 includes a media drive for reproducing data 

a copyright and the prevention of the copying, there are a which have been recorded by driving a data medium such as 

kind of data processing techniques such as data hiding, data an optical disk, and more specifically, a CD-ROM drive or 
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DVD (Digital Versatile Disk) drive or a storing media such ID data insert circuit 120. Here, it is desirable that the IC 

as a semiconductor memory device. card 110 is arranged so that the ID data stored by the 

As shown in FIG. 6, the data receiving apparatus 101 has memory 122 is prevented from being easily changed by a 

an interface 112 for connection with the network 102, an user's operation, or so that the algorithm used by the ID data 

interface 118 for connection with the local media 105 and a * insert circuit 120 is prevented from being easily analyzed by 

card interface 111 for connection with the IC card 110. Data a user's operation (tamper resistance module structure), 

supplied from the network 102 via the interface 112 are The data receiving apparatus 101 transfers data (digital 

supplied to a first input terminal of a selector 115 via a filter data into which ID data have been inserted) outputted from 

113 and a descramble circuit 114. Data supplied from the the IC card 110 via the card interface 111 into the output 

local media 105 via the interface 118 are supplied to a *0 circuit 117. The output circuit 117, as described above, 

second input terminal of the selector 115. Data supplied converts the digital data into which the ID data have been 

from the IC card 110 via the card interface 111 are supplied inserted by the ID data insert circuit 120 into an analog 

to the monitor 104 via an output circuit 117. A controller 116 signal or a composite video signal with respect to the 

is connected to the card interface 111, and the selector 115 television receiver. 

is controlled by the controller 116. 1S The following describes the operation of the present 

The filter 113 is a circuit for extracting predetermined embodiment with reference to the flow chart in FIG. 8, 

control data from multiplexed data transmitted via the The data reproducing apparatus of the present embodi- 

network or necessary data from a multiprogram. The ment is arranged so as to be operable by connecting the IC 

descramble circuit 114 performs a descramble process (a card 110 to the receiving apparatus 101 (step SI). Namely, 

kind of decrypt process) for reproducing data which have 20 when the controller 116 of the receiving apparatus 101 

been subject to predetermined scramble process (a kind of detects the connection of the IC card 110 via the card 

encrypt process) to an original state. interface 111, it starts to receive and reproduce data based on 

The selector 115 is a selecting circuit for selecting one of the user's ID data from the IC card 110. The IC card 110 is 

received data from the network and output data from the supplied only to a user who contracts with the data service 

local media 105 under control of the controller 116. The provider in the data transmitting apparatus 103 side, and the 

controller 116 is a control apparatus of the receiving appa- ID data for identifying the user are previously stored into the 

ratus 101, and it recognizes the received data from the memory 122. According to the present embodiment, as the 

network or the output data from the local media 105 through, data transmitting apparatus 103, a digital broadcasting net- 

for example, a switch, not shown, provided to the receiving work which always originates multimedia data such as an 

apparatus 101, and controls the selector 115 according to the image to many and unspecified users, or a system of 

recognized result. Here, as to the method of controlling the providing multimedia data (video on demand) such as a 

selector 115, besides the aforementioned one, a method of movie according to a request from the user via the computer 

first selecting received data from the network may be used network is assumed. 

in the case of stopping the local media 105, or the controller 35 When the IC card 110 is mounted to the card interface 111 

116 may control the controller 115 according to a selecting by the user, the controller 116, as described above, recog- 

signal from a personal computer or a television receiver as nizes as to whether the data to be reproduced are the 

the monitor 104. received data from the network 102 or the output data from 

The output circuit 117 includes a D/A converting circuit the local media 105 (step S2). Here, it is assumed that the 

for converting digital data, which is the receiving data from 4Q case where the received data from the network 102 are 

the network or the output data from the local media, into an reproduced (YES of step S2). 

analog signal, or a CVS converting circuit for converting the The data receiving apparatus 101 receives data transmit- 

digital data into a composite video signal which is supplied ted from the data transmitting apparatus 103 via the network 

to the television receiver, they converts provided data into a 102 at the interface 112, and the data goes through signal 

reproducing signal for reproduction on the monitor 104. The 45 processing systems such as the filter 113 and the descramble 

card interface 111 includes a card slot for connection with circuit 114 so as to be transmitted to the selector 115. The 

the IC card 110, described later, the card slot connects the IC controller 116 outputs the received data (digital image data, 

card 110 with the controller 116, selector 115 and the output etc.) from the data transmitting apparatus 103 selected by the 

circuit 117 in the receiving apparatus 101. selector 115 to the ID data insert circuit 120 of the IC card 

The IC card 110 is prepared for each user, and it contains, 50 HO via the card interface 111 (step S3), 

as shown in FIG. 7, a microprocessor (CPU) 121 which In the IC card 110, the ID data insert circuit 120 inserts the 

forms the controller, a memory 112 and a data insert circuit ID data stored in the memory 122 into the digital image data 

120. The CPU 121 is connected to the controller 116 in the which are data outputted from the receiving apparatus 101 

receiving apparatus 101 side via the card interface 111, and according to control of the CPU 121. 

it outputs, for example, user ID data, and controls the ID data 55 One example of the data insert circuit 120 is shown in 

insert circuit 120. FIG. 9. This example is a circuit for inserting the ID data into 

The ID data insert circuit 120 is a circuit dedicated to digital image data which has been encoded by MPEG 
concealedly inserting ID data previously set per user into the method. The MPEG data is divided into two data; one is for 
data outputted from the selector 115 via the card interface describing the image itself (such as pixel data, motion vector 
111. Namely, the ID data insert circuit 120 performs data 60 data) and the other. FIG. 9 shows an example in which the 
processing algorithm which is called as the conventional ID data is inserted into the pixel data, 
data hiding, data insert, digital watermarking or digital Source contents supplied from the card interface 111 is 
fingerprint, and the ID data insert circuit 120 concealedly input to an input data analyzer 92 in which a data structure 
inserts, for example, ID data for identifying a user and stored of a data block is analyzed in two steps. First, it is deter- 
in the memory 122 into provided data. The memory 122 is, 65 mined whether or not the data block is the MPEG data block, 
for example, a rewritable ROM (Read Only Memory), and If the data block is the MPEG data block, then, it is 
is accessed by the CPU 121 and supplies the ID data to the determined whether or not the data block is the pixel data 
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block. If the data block is the pixel data block, the data block Meanwhile, when the output data from the local media 

is so stored in a memory block 94 that it can be separated in 105 is reproduced, the controller 116 outputs the digital 

a unit of one expression value (a luminance value or a image data, etc. outputted from the local media 105 by the 

chrominance value of one pixel) of one expression unit (e.g., selector 115 into the IC card 110 via the card interface 111 

one pixel) and it can be accessed by an input data processor 5 ( s tep S7). The local media 105, as described above, is a drive 

96 and a controller 95 in a unit of one expression value. f or reproducing a movie image and a still image which have 

The memory block 94 stores the result of analysis which been previously recorded in an optical disk, etc. so as to 

can be read by the input data processor 96 and the controller output them. Here, the data processing after transmission to 

95. The controller 95 controls the ID data insert circuit 120 me j C card uq ^ me same ^ tne 0 f th e reproduction 

as a whole and interfaces with other circuits. Functions of 10 of data from ^ nctwork ( step 34 through step S6). 

the controller 95 include a control of start/stop of the j- * iL 

, , n * 1. w ., a twvl As described above, according to the present 

operation of the input data analyzer 92, a read/write control ... ' . j ,™ 

* . 1 1 rxjt j no • * ** * *u • * embodiment, the IC card 110 is prepared for each user, the 

of the memory blocks 94 and 98, an instruction to the input r. . * r . . 

3 n , . t . * * ( a *u a* ID data for identifying the user are concealedly uiserted mto 

data processor 96 relaUng to the insert data and the data 7 ^ . ■ 3 , , Tn 

; ... j j * • »■ r rt , m j„* b the provided data such as the digital image data by the ID 

insert position, and data processing operation tor ID data , *\ i->ak -i. • *u ir* ^11 a o-J^ : f 

« # « ui 1 no a il,. .u,,. tu,, 15 data insert circuit 120 built in the IC card 110. Therefore, if 

stored .n he memory block 98 As described above, the reproduced data into a recording medium 

memory block 98 stored the ID data and distributes it illegally, the user who copied me data can 

The ID data is read by the controller 95 and supplied to be ified fc ducin me user>s ro data and mus the 

the input data processor 96 after being processed such as a * ies caQ be eliminated . 

encrypted. The controller 95 instructs the input data proces- , 0 * r 

sor 96 of the ID data and the insert position. The input data u Only when the user mounts the IC card lit supplied upon 

processor 96 reads data from the memory block 98 and the contraction with the data service provider to the data 

processes it. The position of data to be processed is receiving apparatus 101, the user can reproduce desired 

instructed by the controller 95. The controller 95 accesses multimedia data using a personal computer and a television 

the memory block 98 and scans the data region which is „ >f«wer without^ being conscious of the process for inserting 

determined as the pbrel data block by the input data analyzer * »e ID data. Therefore, when the data service provider 

92. The controller 95 determines the insert position using a P™}*' 5 data ™ a "^P^ network aD <? * broa ^ 

function f which can uniquely determine the insert position castin g ™ tw °*' ^ P rocess for msertng ue ID data can be 

based on a parameter which is a pixel value obtained by the eliminated. The data service provider does not have to 

above scanning. The determined insert position is supplied 30 P«fonn the data inserting process. As a result^ like the 

, ,. • , , ,„ oc J computer network and digital broadcasting network, the 

to the input data processor Vo. v . B . . ... A . . 

_ . , . j j , c „ „,„ „ construction of the data service system for providing data to 

The input data processor 96 reads data from the memory , ... ' u -i i-j 

. , i «d j _r j . • . • "f *u j , many and unspecified users can be easily realized. 

block 98 and performs a data insertion operation it the data ' , . . . , ,. , 

unn/iiauupu r , Therefore, by the ID data inserting process, the protection of 

is one expression unit (one pixel) and does not if U is not. .J^ ^ data ^ ^ ^ 

Theoulput of the input data processor 96 is supplied to the 35 ; * ed and , he of distribution of the data 

card interface 111. Data insertion mto one expression unit *\ , . , J 

, .v . - i r 11 x ■ if service can be improved, 

(one pixel) is performed as follows. An expression value tor * 

one bit is inserted into one expression unit of data. If the In addition, also when data are reproduced from the local 

expression value for one bit is 0, the data is not changed and media 105, it is not necessary to provide the ID data insert 

is supplied to the card interface 111 as it is. If the expression 40 function to the local media 105. Therefore, the rights such as 

value for one bit is 1, that expression value is replaced with * copyright of the data providers who provide data via the 

an expression value which is most similar to the expression local media 105 can be protected without preparing special 

value of an object to be processed and the replaced value is l° ca l media 105. 

output to the card interface. There is defined a relation Here, the present embodiment refers to the function for 

between the expression value for each expression block and 45 inserting the ID data into data in the IC card 110, but also 

at least one (not all) possible value of the expression value. specific data representing a copyright of data creators and 

The replacement of the expression value means that at least providers may be inserted thereinto. Moreover, besides the 

one possible value of the expression value for each expres- user's ID data, for example, date, specific sort data, etc. may 

sion block is selected based on this relation. In one practical be inserted thereinto. 

example, the luminance value or chrominance value of the 50 It is possible to supply the output of the output circuit 117 

object pixel is replaced with at least one luminance value or to a storing device of multimedia data in stead of a repro- 

chrominance value having a minimum difference value ducing device of multimedia data, such as the monitor 104. 

(absolute value) with the luminance value or chrominance Further, it is possible to output digital data before D/A 

value of the object pixel. If there are plural expression values conversion from the output circuit 117 in stead of analog 

which are most similar to the expression value of the object 55 data. These modifications can be applied to the following 

pixel, one having the minimum absolute value is selected. embodiments. 

Returning to FIG. 8, the IC card 110 supplies the digital . 

image data, etc. into which the ID data have been inserted to lllird Embodiment 

the output circuit 117 of the data receiving apparatus 101 via The following describes the first modified example of the 

the card interface 111 (step S4). 60 second embodiment as the third embodiment. As shown in 

The output circuit 117 converts the multimedia data such FIG. 10, in the third embodiment, a descramble circuit 123 

as the digital image data into which the ID data have been is added as a component of the IC card 110 in the second 

inserted into an analog signal or a composite video signal so embodiment. In this case, needless to say, the descramble 

as to output the signal into the monitor 104 (step S5). On a circuit 114 in the data receiving apparatus 101 can be 

display screen of the television receiver, for example, which 65 omitted. Therefore, the arrangement of the data receiving 

is the monitor 104, an image transmitted from the network apparatus 101 can be simplified, and the cost can be reduced, 

is reproduced (step S6). Moreover, by providing the descramble circuit 123 to the IC 
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card 110 side, when scrambled data are provided from the 
data transmitting apparatus 103, the user without the IC card 
110 cannot release the scramble, and thus the rights of the 
data service provider can be protected more efficiently. The 
scramble algorithm can also be easily updated by merely 
replacing the IC card 110. Here, the descramble circuit may 
be double-arranged by the descramble circuit 114 provided 
to the data receiving apparatus 101 and the descramble 
circuit 123 of the IC circuit 110, namely, the descramble 
circuit 114 in the data receiving apparatus 101 side is not 
omitted. 

Fourth Embodiment 

In the second embodiment, the ID data insert circuit 120 
performs the data insert process for concealedly inserting ID 
data according to data processing algorithm previously 
stored into the IC card 110. On the contrary, the data 
processing algorithm is not necessary to be previously stored 
in the IC card 110, and algorithm for the data insert process 



data through the descramble circuit 114. At this time, the 
security controller 150 of the IC card 110 decrypts the 
scrambled key data from control data which are separated 
from the received data by the filter 113, and outputs the 
scramble key data to the descramble circuit 114. The 
descramble circuit 114 decrypts the original data from the 
scrambled data using the scramble key data supplied from 
the IC card 110. 

In such a user's identification process, the security con- 
troller 150 of the I C card 110 decrypts the scramble key data 
using the ID data corresponding to a master key data set per 
card (namely, per user). Namely, the data transmitting appa- 
ratus 103 encrypts the scramble key data according to the 
master key data, and multiplexes the data into data to be 
transmitted. 

According to the present embodiment, as shown in FIG. 
11, after the IC card 110 is connected to the data receiving 
apparatus 101, the security controller 150 supplies the ID 
data necessary for the user's identification process (control 



may be included in data which are received by the data 20 0 f the descramble circuit 114) to the ID data insert circuit 



receiving apparatus 101 from the data transmitting apparatus 
103, and the IC card 110 may download also the algorithm 
for the data insert process. The following describes the 
fourth embodiment having the above arrangement. 
However, since the arrangement of the fourth embodiment is 25 
the same as that of the second embodiment, the detailed 
description is omitted. 

Namely, the CPU 121 of the IC card 110 shown in FIG. 
7 receives the algorithm for the data insert process from the 



120. Namely, after the security controller 150 performs the 
user's identification process and identifies that the user is 
accepted as the data receiving person, the security controller 
150 supplies the ID data to the ID data insert circuit 120. 
However, before the user's identification process, the secu- 
rity controller 150 may supply the ID data to the ID data 
insert circuit 120. In this case, needless to say, when the user 
is not accepted as the data receiving person according to the 
identification process, the data cannot be 



data receiving apparatus 101, and downloads it to the 30 descrambled. Therefore, it becomes meaningless that the ID 



35 



memory 122 or an internal memory of the ID data insert 
circuit 120. The ID data insert circuit 120 performs the data 
insert process for concealedly inserting ID data according to 
the downloaded algorithm for the data insert process. 

In this method, the algorithm for the data insert process is 
not fixed, and the method of the data insert process can be 
easily updated and changed as occasion demands. Moreover, 
the ID data insert circuit 120 in the IC card 110, which is 
standardized so that not specific algorithm for the data insert 
process but algorithm set externally is performed, can be 
used. Therefore, the standardization of the IC card 110 
becomes easy, and the data insert process according to 
special algorithm provided by the data provider can be 
performed. Therefore, the rights such as a copyright can be 
protected more efficiently. 

Fifth Embodiment 

FIG. 11 is a block diagram relating to the fifth embodi- 
ment. 50 

The IC card 110 of the present embodiment performs the 
identification process of a user (method limiting a receiving 
person and which is called as conditional access) in the 
conventional data service, so the IC card 110 includes a 
security controller 150. The security controller 150 com- ss 
prises the CPU 121 and the memory 122 shown in FIG. 7 as 
hardware. 

Here, the user's identification process is of identifying a 
user who contracts with the data service provider by using 



45 



data are supplied to the ID data insert circuit 120. 

The ID data insert circuit 120 concealedly inserts the ID 
data supplied from the security controller 150 into data 
supplied from the descramble circuit 114. The security 
controller 150 may supply an algorithm for the insert 
process downloaded from the data receiving apparatus 101 
or an algorithm for the insert process which is previously 
stored in the internal memory to the ID data insert circuit 
120. 

The other arrangement is basically the same as that shown 
in FIG. 6, but is different from that in FIG. 6 in that a 
multiplex signal outputted from the descramble circuit 114 
is separated by a demultiplexer 130 into a digital video 
signal and an audio signal, for example. A detecting circuit 
131 of the audio signal extracts an audio signal from the 
output of the demultiplexer 130 and outputs it to a D/A 
converter 133. A detecting circuit 132 of the video signal 
extracts a video signal from the output of the demultiplexer 
130 and outputs it to a D/A converter 134. The D/A 
converters 133 and 134 convert the video and audio signals 
into analog signals and outputs the analog signal into the 
monitor 104. 

Sixth Embodiment 

As shown in FIG. 12, in the sixth embodiment, the 
descramble circuit 114 which is provided in the data receiv- 
ing apparatus 101 in the fifth embodiment is located in the 
IC card 110, and thus the sixth embodiment corresponds to 



the IC card 110, and more concretely, it is a process for 60 the third embodiment (step See FIG. 10). 



decrypting key data (key data for decrypting scrambled 
encrypted data) necessary for an operation of the descramble 
circuit 114 and outputting them so that the transmitted data 
is descrambled. Namely, the data transmitting apparatus 103 
transmits scrambled (a sort of encrypted) data via the 
network 102 by using the scramble key data. The data 
receiving apparatus 101 decrypts original data from received 



65 



When the algorithm for the scramble process in the data 
transmitting apparatus 103 is updated, accordingly it is 
necessary to update the algorithm of the descramble circuit 
114, but in such an arrangement, only the specification of the 
IC card 110 is changed. In the arrangement that the 
descramble circuit 114 is provided to the data receiving 
apparatus 101, the descramble circuit 114 of the data receiv- 



09/26/2003, EAST Version: 1.04.0000 



6,061,451 

15 16 

ing apparatus 101 should be updated according to the the data provider is not required. The operations of the 

updating of the algorithm for the scramble process, and security controller 150 and the ID data insert circuit 120 are 

actually, the data receiving apparatus 101 itself should be the same as those in the ninth embodiment. 

updated. However, according to the sixth embodiment, this Moreover, according to the present embodiment, the ID 

can be unnecessary. 5 data insert circuit 120 is located in the output side of the 

demultiplexer 130, and the ID data insert circuit 120 inserts 

Seventh Embodiment ID data mto a vidc0 signa] which ^ an outpul of lhc 

As shown in FIG. 13, in the seventh embodiment, the ID demultiplexer 130. However, the arrangement of the present 

data insert circuit 120 which is provided in the IC card 110 embodiment is not limited to this. Namely, the ID data insert 

in the above-mentioned embodiments is located in the data 10 circuit 120 may be located between the descramble circuit 

receiving apparatus 101. The ID data insert circuit 120 114 and the demultiplexer 130, and insert the ID data into 

concealedly inserts ID data supplied from the security data descrambled according to the descramble circuit 114. 

controller 150 in data supplied from the descramble circuit Moreover, the ID data insert circuit 120 may be located in 

2^4 the output side of the demultiplexer 130 and insert the ID 

In this case, the ID data insert circuit 120 may previously 15 into an auto si^al which is an output of the demuU 

store the algorithm for the insert process, or be given with tl P* e * er ™ ( namel ^ be weeD *. e demultiplexer 130 

the algorithm for the insert process by the security controller and the detectm S circuit 131 of aucho sl S nal >- 

150. Moreover, the algorithm for the insert process may be meveQth Embodiment 

downloaded from the data transmitting apparatus 103 via the 2Q 

data receiving apparatus 101 to the security controller 150, Multimedia data are communicated by multiplexing video 

and then may be supplied to the ID data insert circuit 120. data and audio data according to the system specification of 

Further, the algorithm for the insert process is previously the MPEG -2, for example, as a stream data which is a coded 

stored in the data receiving apparatus 101, and is down- (compression-coded) bit stream (stream data). Where the 

loaded to the security controller 150. Thereafter, the algo- 25 data insert process (hereinafter, referred to as FP process as 

rithm may be supplied to the ID data insert circuit 120. The an abbreviation of the fingerprint process) in the user's side, 

other arrangement is the same as that in the aforementioned normally coded stream data is subject to the decode process 

fifth embodiment. (expansion-decoding process) before inserting a mark. 

c • u u n u a • In an electronic watermark printing, data which is inserted 

Eighth Embodiment 3Q ^ stream daU ig cal]ed ^ « mark » mark mav be an 

As shown in FIG. 14, in the eighth embodiment, the identification data for specifying the user, a copyright, or 

descramble circuit 114 which is provided in the data receiv- other information or other data. The process for inserting a 

ing apparatus 101 in the seventh embodiment is located in mark which identifies the user is called a fingerprint pro- 

the IC card 110. Therefore, the data receiving apparatus 101 cessing. 

transfers data received from the interface 112 to the 35 The reason why the coded stream data is decoded before 

descramble circuit 114 of the IC card 110, data are mark insertion, is that it is difficult to determine a suitable 

descrambled therein and are outputted thereinto. The ID data position into which a mark is inserted in a coded bit stream, 

insert circuit 120 concealedly inserts ID data supplied from Namely, when a mark is inserted into a stream data by the 

the security controller 150 into data supplied from the FP process, image quality of contents data is deteriorated 

descramble circuit 114 of the IC card 110. The other 40 depending on an insert position. Moreover, a position where 

embodiment is the same as that of the aforementioned fifth a mark cannot be inserted by the FP process according to 

embodiment. syntax exists in the stream data. Therefore, when a mark is 

inserted into a stream data by the FP process, normally, a 

Ninth Embodiment mark is inserted into a decoded stream data, and the stream 

As shown in FIG. 15, in the ninth embodiment, the ID 45 data mt0 which the mark has been 15 sub J ect to the 

data insert circuit 120 which is provided in the ID card 110 compression-coding process again. 

in the eighth embodiment is located in the output side of the However, at the time of such a mark insert by the FP 

demultiplexer 130. Namely, the ID data insert circuit 120 process after decoding, it is difficult to realize a real-time 

concealedly inserts ID data supplied from the security property. Therefore, where multimedia data are provided at 

controller 150 into video data which have been obtained by 50 real time such as the case of a movie, due to an influence of 

separating a multiplex signal by the demultiplexer 130. the process of inserting a mark into the stream data, it 

According to such an arrangement, the ID data insert becomes difficult to provide the multimedia data at real time, 

circuit 120 does not require a function for detecting an image Therefore, the following describes an embodiment of the 

from data in which a sound and an image are multiplexed, 5S data receiving apparatus which realizes the real-time prop- 

so it can concealedly insert ID data into the image data erty of the FP process and in which the mark insert does not 

efficiently (at a high speed). The other arrangement is the affect the quality of stream data as the eleventh embodiment. 



same as that in the fifth embodiment. 

Tenth Embodiment 



Eleventh Embodiment 



60 FIG. 17 is a block diagram of the data receiving apparatus 

As shown in FIG. 16, in the tenth embodiment, the according to the eleventh embodiment, and FIG. 18 is a 

security controller 150 and the ID data insert circuit 120 are block diagram of a fingerprint circuit 206 in the data 

located in the data receiving apparatus 101 without using the receiving apparatus 201 in FIG. 17, and FIG. 19 is a flow 

IC card 110 in the ninth embodiment. chart explaining an operation of the present embodiment. 

According to this embodiment, the data receiving appa- 65 As shown in FIG. 17, as the eleventh embodiment, there 

ratus 101 should be different per user, but there is an is provided a data service system in which a data transmit - 

advantage that an operation for managing the IC card 110 by ting apparatus 202 is connected to a data receiving apparatus 
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201 via a communication medium 203 of a computer 
network or a digital broadcasting network, for example. The 
data transmitting apparatus 202 is an apparatus located at a 
side of data provider and transmitting multimedia data 
obtained by multiplexing video and audio data according to 
contents of contraction or a user's request. Hereinafter, the 
data transmitting apparatus 202 is occasionally referred to as 
a general idea including the data provider. Meanwhile, the 
data reproducing apparatus 209 is a user's home apparatus 
(corresponding to a set top box or IRD: Integrated Receiver 
Decoder) provided to the user's side, and it performs a 
process, described later, on received multimedia data, and 
reproduces and outputs contents data included in the mul- 
timedia data (hereinafter, occasionally described as a general 
idea including the user). 

As shown in FIG. 17, the data receiving apparatus 201 
includes an interface 204, a descramble circuit 205, an FP 
circuit 206, a decoder 207 and an interface 208. The inter- 
face 204 receives a stream data in the form of bit stream 
transmitted via the communication medium 203. The 
descramble circuit 205 performs the descramble process (a 
kind of decrypt process) for reproducing an original state of 
data prescriptively descrambled (a kind of encrypt) by the 
data receiving apparatus 202. In the case of receiving data 
which have not been scrambled, and in the case of perform- 
ing the descramble process using another apparatus, the 
descramble circuit 205 may not be provided. 

The FP circuit 206, as described later, receives a stream 
data coded by the MPEG-2 system, for example, and per- 
forms the FP process for inserting a mark (data, etc. for 
specifying a user) for protecting rights such as a copyright 
based on predetermined algorithm. The algorithm, as men- 
tioned before, is a method for realizing the data insert 
process such as the digital watermarking and data hiding. 

The FP circuit 206 performs the FP process for inserting 
a mark into a predetermined position of the coded stream 
data (a mark insert position, described later), and outputs the 
stream data into the decoder 207. The decoder 207 is a 
component for decoding the stream data coded by the 
MPEG-2 system, for example. The interface 208 is a signal 
converting section for converting the decoded stream data 
into a signal according to the specification of the data 
reproducing apparatus 209. 

The data reproducing apparatus 209 is a personal com- 
puter and a television receiver (TV) having a display device. 
The interface 208 includes a D/A converter for converting 
digital data into an analog signal or a CVS converter for 
converting digital data into a composite video signal for the 
television receiver. Moreover, the data reproducing appara- 
tus 209 includes not only the display device but also an 
output device such as a DVD (Digital Versatile Disk) drive. 
The DVD drive receives digital data outputted from the 
interface 209, and records the digital data to the DVD as a 
recording medium. 

FIG. 18 is a detailed block diagram of the FP circuit 206. 
The FP circuit 206 has an input buffer 220, a syntax analysis 
circuit 221, a mark insert circuit 222, an output buffer 223, 
an insert position determining circuit 224, a mark insert 
controller 225, a parameter memory 226 and an insert 
position history memory 227. 

The input buffer 220 temporarily stores a stream data in 
the form of a bit stream descrambled according to the 
descramble circuit 205, and outputs the stream data into the 
syntax analysis circuit 221. The syntax analysis circuit 221 
analyzes a syntax (a rule of coded stream data) relating to a 
coding characteristic and a data structure from the stream 
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data, and outputs the syntax data CD. The coding charac- 
teristic means characteristic data such as VLC (variable 
length code), FLC (fixed length code) used for the coding, 
and a data position not to be skipped. The data structure 

5 means specific data such as a sequence layer, a GOP (Group 
of Pictures) layer, a picture layer, a slice layer, a microblock 
layer, a block layer which are hierarchical structures of 
image data in the coding/decoding method according to the 
MPEG-2, and a data element (data item) set per each layer 

to of hierarchical structure. Namely, the syntax is a rule rep- 
resenting the order and position of the data element (block 
of significant bit stream) before and after coding. Therefore, 
the syntax analysis circuit 221 determines a position, type 
and value of the data element forming the stream data, and 

15 output them as syntax data CD. The syntax analysis circuit 
221 comprises a software operated by the CPU or a special 
hardware (logic circuit, etc.). 

The insert position determining circuit 224 determines a 
mark insert position in the stream data where a mark is 

10 inserted and a mark based on the syntax data CD, marking 
parameter data stored in the parameter memory 226, and 
mark insert history data stored in the insert position history 
memory 227. The determined data is insert position data PD. 
Examples of the marking parameter data in the parameter 

25 memory 226 are user specific data for identifying a user, use 
terminal specific data for specifying a user's terminal (data 
receiving apparatus 201), use service specific data for speci- 
fying contents of use service, use area specific data for 
specifying a use area, use time data for a time the user uses 

30 the provided data, predetermined common secret data, ran- 
dom data, marking parameter parity data, marking parameter 
completeness data. Further, the insert position history 
memory 227 records data representing a mark insert position 
where a mark is actually inserted into the stream data by the 

35 mark insert controller 225. 

The mark insert controller 225 controls the mark insert 
circuit 222, and sets a mark insert position in the stream data 
and a content of a mark to be inserted according to the insert 

40 position data PD outputted from the insert position deter- 
mining circuit 224. The mark insert circuit 222 performs the 
FP process for inserting the set mark into the insert position 
in the stream data transmitted via the syntax analysis circuit 
221. The output buffer 223 temporarily stores the stream 

45 data after the mark has been inserted by the mark insert 
circuit 222, and outputs it to the decoder 207. 

The operation of the present embodiment will be 
described with reference to the flow chart shown in FIG. 19. 
According to the present embodiment, a data providing 

50 service system which, provides multimedia data such as a 
movie from the data provider 202 side to the contract user 
201 via the digital broadcasting network, etc. is assumed. 
The data transmitting apparatus 202 transmits a stream data 
formed of a bit stream coded by the syntax of the MPEG-2 

55 system specification, for example, including contents data 
such as a movie (composite data of an image and a sound) 
(step Sll). The data receiving apparatus 201 receives the 
stream data transmitted via the interface 204, and 
descrambles the stream data using the descramble circuit 

6Q 205 (steps S12 and S13). 

Next, according to the present embodiment, the FP circuit 
206 performs the FP process for inserting a mark which is 
specific data for protecting rights such as a copyright of the 
multimedia data into the determined position of the stream 

65 data (steps S14 through S16). Namely, as shown in FIG. 18, 
in the FP circuit 206 of the present embodiment, the syntax 
analysis circuit 221 analyzes the syntax relating to the 
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coding characteristic and the data structure, and outputs the mark insert position necessary for the FP process without 

syntax data (step S14). decoding the coded stream data, and can perform the mark 

Further, the insert position determining circuit 224 deter- insert process. Since the mark can be inserted to the suitable 

mines the mark insert position of the stream data where a position in the stream data without decoding and again 

mark is inserted and a mark based on the syntax data CD, the s coding the stream data, the mark insert process can be 

marking parameter data stored in the parameter memory performed at real time. 

226, and the mark insert history data stored in the insert In addition, when as the mark data not only the ordinary 

position history memory 227 (step S15). The determination user specification data but also lime stamp data, terminal 

of mark insert position is specification of a block in the data specification data, area specification data, etc. are prepared 

structure of the MPEG-2, for example, and selecting of the 10 in the parameter memory 26 so as to be used as the mark 

block which has weak possibility of being skipped. In the data, there also produces an effect that evidence which 

stream data, the syntax differs with the adopted coding and proves an infringement of digital rights is strengthen, 

decoding method such as the MPEG-2. Therefore, where a Moreover, when parity and hash data are included in the 

mark is inserted by the FP process, an influence which is mark itself or parameter data, it is possible to easily realize 

exerted on the contents data (user data which are multimedia 1S detection of alternation of a mark. 

data used by a user) also differs with the insert position. In the case the stream data is image data in the process for 

Stated another way, a degree of deterioration in an image determining the mark insert position according to the present 

upon reproduction differs. Moreover, a position where a embodiment, considering a visual influence of the image, a 

mark cannot be inserted also exists depending on the syntax. method of superposing a characteristic pattern of the image 

Therefore, the insert position determining circuit 224 can 2 q (a peak of DCT coefficient, etc.) on the image data is 

determine a suitable mark insert position based on the syntax considered. The DCT means Discrete Cosine Transform. In 

data relating to the coding characteristic and the data struc- this case, a method of selecting a most remarkable peak is 

ture obtained by the syntax analysis of the coded stream considered. Moreover, a method of detecting one or some 

data. peaks in the image from a plurality of candidates produces 

The coded stream data is in accordance with a predeter- 2 5 an effect that the specification of a mark is made difficult and 

mined syntax. For example, MPEG-2 video data is in the alternation of the mark is prevented. Further, a method 

accordance with an international standard ISO 13818-2. The of selecting the mark insert position from a plurality of 

video stream data is represented by a concatenate of data candidates at random is effective in that the specification of 

elements. The data element comprises a header code such as the mark position is made difficult. In this method, the mark 

a sequence header code and a group start code header, and 30 insert position specifies the position of the data element 

character code such as a horizontal size value, a vertical size (DCT coefficient) determined by the mark insert algorithm, 

value, an intra quantizer matrix and a DCT DC size lumi- In addition, there assumes injustice such that in order to 

nance. The syntax analysis in MPEG-2 means an analysis of remove or cancel the inserted mark, when after the stream 

video stream in accordance with the MPEG-2 video syntax data into which the mark has been inserted is decoded, it is 

in order to know what order or sequence of the data elements 35 again compressed by shifting or sampling a frame, the 

is linked in the video stream. The syntax data represents stream structure (data structure) is changed so that the mark 

what data element is located in which position (bit position) is altered (for example, a previous P frame becomes a new 

in the video stream. For example, the syntax data represents I frame). Even if the previous P frame is not a frame into 

that the sequence header code data are located from the first which a mark has not been directly inserted, when a mark 

to thirty second bits and the horizontal size value data are 40 has been inserted into a just previous I frame, an influence 

located from the thirty third to forty fourth bits. Further, the 0 f the mark remains as a result of the predictive coding, 

syntax data may includes the value of the bit stream of the Therefore, since the influence of the mark remains even after 

horizontal size value. The mark insert position represents the recompressing, the mark can be reproduced by detecting 

into which data element the mark is inserted, i.e., a specified the influence. Moreover, it is desirable that the mark is 

data element. 45 spatially inserted into one I frame or that the mark is inserted 

The mark insert circuit 222 performs the FP process for into a plurality of I frames. Further, it is desirable that the 

inserting a mark into a determined insert position in the mark is inserted into the front and last microblock of a slice 

stream data according to control of the mark insert controller layer in the image data of the MPEG-2, for example, or that 

225 based on the insert position data PD (step S16). The the mark is inserted into an arbitrary microblock. 

decoder 7 performs the decoding process on the stream data 50 As described above, according to the present 

which has been subject to the FP process and outputs the embodiment, the data, which are transmitted as the stream 

stream data into the interface 208 (step S18). The interface data formed of the bit stream coded by a predetermined 

208 performs the signal converting process for converting coding method, is received via a computer network or a 

the stream data into a signal according to the specification of digital broadcasting network, and the syntax analysis pro- 

the data reproducing apparatus 209 connected to the data 55 cess is performed on the stream data, and the syntax data 

receiving apparatus 201. relating to the coding characteristic and data structure of the 

According to the present embodiment, the FP process for stream data are obtained, and the insert position in the 

inserting the mark which is the specific data for protecting stream data, into which a mark which can be reproduced is 

rights such as a copyright of the data provider into the concealedly inserted, is determined based on the syntax data 

provided stream data can be performed by the FP circuit 206 60 obtained by the syntax analysis process and predetermined 

of the data receiving apparatus 201 which is the user side. parameter data which are prepared. Therefore, in the user's 

When the syntax analysis process is performed on the stream data receiving apparatus side, the FP process for inserting a 

data coded by the MPEG-2 system specification, for mark for protecting the rights such as a copyright into a 

example, and syntax data relating to the coding character- suitable position determined based on the syntax data relat- 

istic and data structure are obtained, the suitable mark insert 65 ing to the coding characteristic and data structure can be 

position in the stream data into which the mark is inserted is performed on the stream data provided by the data provider, 

determined. As a result, the FP circuit 206 can determine the Therefore, the FP process can be performed at real time 
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without previously decoding the coded stream data. In other The conversion table generating circuit 228 does not have 

words, the processes for decoding and compression- to always generate the conversion table, so it may outputs 

decoding for the FP process for mark insertion can be conversion table data which are prepared by default. The 

omitted. Moreover, since it is not necessary to perform the other components are the same as those in the eleventh 

coding process on the stream data into which the mark has 5 embodiment. 

not been inserted, it can be prevented that a compression Thirteenth Embodiment 

factor is lowered and thus a quantity of data is increased. M desM ^ ^ ^ ag & copyr - ght of 

Though the FP circuit 206 is described as a hardware, it multimedia data can be protected by the FP technique, and 

may comprise a memory which holds a fingerprint process- tmis rea lization of the data providing service system 

ing program formed of the aforementioned algorithm and a ™ becomes easy. However, the multimedia data are usually 

microprocessor (CPU) for performing the FP processing communicated as stream data obtained by multiplexing 

program. video data and audio data according to the MPEG-2 system 

rri i p,t w-t l j * * specification, for example. As to the stream data, since 

Twelfth Embodiment r 4 / , , \ , . , ,.~ / 

syntax (a rule of coded stream data) differs with the coding/ 

FIG. 20 is a block diagram showing a configuration of the 15 decoding (CODEC) method adopted, where a mark is 
FP circuit 206 according to the twelfth embodiment relating inserted by the FP process, an influence which exerts con- 
to a modification of the eleventh embodiment. In addition to tents data (user data which are multimedia data to be used by 
the arrangement of the FP circuit in the eleventh a user) also differs with the CODEC method. More 
embodiment, the FP circuit 206 according to the present specifically, the FP process causes a deterioration in an 
embodiment, further includes a conversion table generating 20 image of the contents data, and a certain stream has a 
circuit 228 and a conversion table memory 229. The con- position into which a mark cannot be inserted by the FP 
version table generating circuit 228 generates a conversion process according to the syntax. 

table (a table showing a conversion bit stream) for a bit There exist no FP process which is effective in all CODEC 

stream (partial stream), in which the influence of the mark method. Moreover, a fingerprint process for inserting spe- 

insertion exerted on the respective data of the VLC and FLC c ific data not into the contents data of a video (including a 

is considered, based on the coding characteristic of the still image) and audio but into header data in a stream is 

coded stream data obtained by the syntax analysis circuit considered, but the insert position of the specific data is 

221. The conversion table memory 229 stores the conversion easily specified, and thus removal and alternation of the 

table data. ^ inserted specific data are easy. Therefore, it is not a desirable 

The mark insert controller 225 instructs the mark insert method, 

circuit 222 to perform the process for replacing a bit stream The FP process is performed in the data receiving appa- 

corresponding to the mark insert position determined by the ra t U s provided to a user. However, when a CODEC method 

insert position determining circuit 224 with a bit stream into for contents to be used is expected, and a function for 

which a mark or a part of the mark is inserted referring to the 3J performing corresponding FP process is provided into the 

conversion table data in the conversion table memory 229. data receiving apparatus, the FP method is limited. 

Namely, the mark insert controller 225 can obtain current Therefore, another CODEC methods cannot be applied to 

position data of the data element before mark insertion based ihe contents. Moreover, needless to say, it is prevented that 

on the coding characteristic according to the conversion a new CODEC method is flexibly applied to the contents and 

table data. The current position data represents the position 4Q a new FP process. 

of the data elements in the bit stream. The mark insert circuit Therefore, in order to comply with this problem, the 

222 or the mark insert controller 225 instructing the mark following describes the thirteenth embodiment which can 

insert circuit 222 of the mark insertion requires the current certainly cope with a CODEC method and FP process 

position data in order to know the position of the data adopted by the data providing service system, 

element which is a subject of insertion in the stream data 45 FIG 2 \ i s a block diagram showing a data receiving 

when the mark is to be inserted. apparatus 301 of the thirteenth embodiment, and FIG. 22 is 

The mark insert circuit 222 stores the bit stream which has a block diagram showing configurations of a data separating 

not been subjected to the FP process in a buffer. The buffer circuit and FP circuit according to the thirteenth 

has a list structure in which the data elements are elements embodiment, and FIG, 23 is a block diagram showing a 

of the list. The buffer is a FIFO buffer. A sequence number 50 detail of the data separating circuit, 

is assigned to the data element which is detected by the According to the present embodiment, as shown in FIG. 

syntax analysis circuit 221. The buffer stores a pair of the 2 1, the data service system is assumed in which a data 

data element and the sequence number. transmitting apparatus 302 is connected to the data receiving 

This sequence number is supplied from the syntax analy- apparatus 301 via a communication medium 303 of a 

sis circuit 221 to the mark insert position determining circuit 55 computer network or a digital broadcasting network, for 

224 as the syntax data. The mark insert controller 225 example. The data transmitting apparatus 302 is the data 

receives the sequence number for specifying the data ele- providing side (hereinafter, occasionally described as a 

ment of the mark insert object from the mark insert position general idea including the data provider) having a data 

determining circuit 224 and supplies the sequence number to transmitting apparatus which transmits multimedia data 

the mark insert circuit 222. The mark insert circuit 222 <j 0 obtained by multiplexing video and audio data according to 

inserts the mark at a suitable position based on the received contents of user's request and contraction. Meanwhile, the 

sequence number. data receiving apparatus 301 is a user's home apparatus 

Alternately, the mark insert controller 225 may be (corresponding to a set top box or IRD and hereinafter, 

informed by the mark insert circuit 222 of the sequence occasionally described as a general idea including the user) 

number specifying the data element which can be inserted at 65 provided to the user's side, and it performs a process, 

that time and may instruct the mark insert circuit 222 how described later, on received multimedia data, and reproduces 

insert the mark. and outputs contents data included in the multimedia data. 
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As shown in FIG. 21, the data receiving apparatus 301 
includes an interface 304, a descramble circuit 305, a data 
separating circuit 306, a fingerprint circuit 307, a decoder 

308 and an interface 309. The interface 304 receives stream 
data SD transmitted via the network 303. The descramble 
circuit 305 performs a descramble process (a kind of decrypt 
process) for reproduce an original state from data which 
have been scrambled (a kind of encrypt process) by the data 
transmitting apparatus 302. There exist various systems as a 
system of the descramble circuit 305, but since they are not 
directly related to the present invention, the description 
thereof is omitted. In the case of receiving data which have 
not been descrambled, or in the case of performing the 
descramble process using another apparatus, the descramble 
circuit 305 may not be provided. 

The data separating circuit 306 is a kind of a filter for 
extracting FP program SDp for the FP process, described 
later, from stream data SD. Where the stream data SD 
includes contents data SDc and FP program SDp, the data 
separating circuit 306 separates and outputs both of them. 
Moreover, where the stream data SD includes only the FP 
program SDp, the data separating circuit 306 extracts the FP 
program SDp. The data separating circuit 306, as shown in 
FIG. 22, a logic circuit 320 and an ID data memory 321 for 
performing the data separating process. The ID data memory 
321 stores ID data for identifying the FP program SDp 
included in a user data area, for example, of the stream data 
SD. Moreover, the ID data memory 321 also stores ID data 
for identifying the FP program SDp corresponding to pro- 
vided contents data. 

The FP circuit 307, as described above, inserts predeter- 
mined specific data (a mark for protecting rights such as a 
copyright) into the provided contents data based on prede- 
termined algorithm. Here, the mark is data for specifying a 
user, etc. The algorithm, as described above, is a method for 
realizing the data insert process such as digital watermarking 
and data hiding. 

As shown in FIG. 22, the FP circuit 307 of the present 
embodiment includes a memory 323 for loading the FP 
program SDp and a microprocessor (CPU) 322 for perform- 
ing the FP process according to the FP program SDp. The 
memory 323 stores the FP program SDp extracted by the 
data separating circuit 306. The FP circuit 307 outputs data 
(contents data) PD, in which specific data have been inserted 
into the contents data SDc of the stream data SD, to the 
decoder 308. 

The decoder 308 is a component for decoding (including 
expanding) the stream data PD which has been coded 
(including compressing) by the MPEG-2 system specifica- 
tion (CODEC system), for example. The interface 309 is a 
signal converting circuit for the decoded stream data RD 
into a signal according to the specification of the data 
reproducing apparatus 310. The data reproducing apparatus 
310 means a personal computer and a television receiver 
(TV) having a display device, for example. The interface 

309 includes a D/A converter for converting digital data into 
an analog signal or a CVS converter for converting digital 
data into a composite video signal (CVS) for the television 
receiver. Moreover, the data reproducing apparatus 310 
further includes an output device such as a DVD (Digital 
Versatile Disk) drive besides the display device. The DVD 
drive inputs digital data outputted from the interface 309 
thereinto, and records the digital data on the DVD which is 
a recording medium. 

The data receiving apparatus of the present invention, as 
described above, is assumed as an apparatus for 
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reproducing/outputting contents data included in multimedia 
data, but the data receiving apparatus may be an apparatus 
located between a transmitting side and a receiving side such 
as a data relay station. Namely, for example, the user side 
5 uses a personal computer as the data receiving apparatus, 
and after the relay station having the data separating circuit 
306 and the FP circuit 307 according to the present embodi- 
ment performs the FP process on the data from the trans- 
mitting side, the data are transferred to the personal com- 
10 puter of the user side. 

A detail of the data separating circuit 306 is shown in FIG. 
23. The logic circuit 320 has a stream ID data reader 370 and 
a stream output selector 371. The stream ID data reader 370 
extracts stream ID data from a received packet and stores it 
15 in the ID data memory 321. The whole stream data SD 
comprises a packet string. 

The stream ID data are ID data for separating contents 
data SDc and corresponding FP program SDp from the 
stream data SD transmitted from the data provider side. The 
20 stream ID data, as shown in FIG. 23, are a table 372 roughly 
comprising a stream identifier, a contents type (for example, 
a type of CODEC system, PID, etc.), and designation data. 

The stream output selector 371 identifies a contents type 
based on an identifier of the received stream data SD 
25 referring to the table 372 stored in the ID data memory 321, 
and separates the contents data SDc and FP program SDp 
based on the contents type and outputs them. The ID data 
memory 321 comprises a non-volatile EEPROM, for 
example, and every time when new stream ID data are 
30 transmitted, they are added and updated. 

The following describes an operation of ,the present 
embodiment with reference to the flow chart shown in FIG. 
24. 

35 According to the present embodiment, a data providing 
service system, which provides multimedia data such as a 
movie to the contract user 301 from the data provider 302 
side via the digital broadcasting network, for example, is 
assumed. The data transmitting apparatus 302 transmits the 

40 stream data SD including contents data (composite data of 
an image and sound) SDc such as a movie and the FP 
program SDp (step S21). The stream data SD is a stream 
data coded by syntax of the MPEG-2 system specification, 
for example. The FP program SDp is a program formed of 

45 algorithm for executing the FP process adapted for the 
syntax. More specifically, the fingerprint process is depend 
on the type of the coding method of the data stream. When 
the type of the coding method changes, the syntax also 
changes so that the mark insert algorithm suitable for the 

so coding method should be searched. 

The data receiving apparatus 301 receives the transmitted 
stream data SD through the interface 304, and performs the 
descramble process by using the descramble circuit 305 
(steps S22 and S23). 

55 Next, the data separating circuit 306 separates the FP 
program SDp and contents data SDc from the stream data 
SD (step S24). Where the stream data SD does not include 
the Fp program SDp, since the FP process, described later, 
cannot be performed, the data reproduction/output process is 

60 stopped as the error process (if NO at step S25, step S30 is 
performed). The error process may includes a process for 
requesting the data provider 302 side to transmit the FP 
program SDp, for example. 

As shown in FIG. 22, the FP circuit 307 loads the FP 

65 program SDp extracted by the data separating circuit 306 
into the memory 323 (if YES at step S25, step S26 is 
performed). The FP circuit 307 performs the FP process for 
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inserting specific data into the contents data SDc separated is transmitted in one packet as a PES (Packetized Elemen- 

by the data separating circuit 306 using the loaded FP tary Stream) packet of the MPEG-2 system specification, for 

program SDp (step S27). The FP circuit 307 has a function example. 

for judging as to whether corresponding FP program SDp On the contrary, in the fifteenth embodiment, it is 

has been loaded into the memory 323, and when loaded, the 5 assumed that the FP program SDp is transmitted as a PES 

process for loading the FP program SDp is omitted, and the packet different from the contents data SDc, and a plurality 

loaded program SDp may be used. of FP programs SDp corresponding to plural types of 

The decoder 308 decodes (including expansion process) contents data are transmitted as an MPEG-2-TS (Transport 
the contents data PD which have been subject to the FP Stream) packet of the MPEG-2 system specification, 
process and outputs the contents data PD into the interface 10 As shown in FIG. 26, in the FP circuit 307 of the present 
309 (step S28). The interface 309 performs the signal embodiment, a program table formed of a plurality of FP 
converting process for converting into a signal according to programs separated by the data separating circuit 306 is 
the specification of the data reproducing apparatus 310 stored in the memory 323 for loading the FP program SDp. 
connected to the data receiving apparatus 301 (step S29). ID data (#0 through #n) for identifying corresponding con- 
More specifically, if the data reproducing apparatus 310 is, ]S tents data are added to the respective FP programs in the 
for example, a personal computer having a display device, program table 323. The ID data may specify PID (Packet 
the data reproducing apparatus 310 converts the decoded Identification) or stream ID data of the stream data including 
stream data (here, contents data such as a movie) RD into an target contents data or may specify both of them, 
analog signal, or outputs a digital signal. The following describes an operation of the fifteenth 

As described above, according to the present 20 embodiment with reference to the flow chart in FIG. 27. 

embodiment, in the data receiving apparatus 301 having the First, as shown in FIG. 26, it is assumed that in the FP 

function for performing the FP process, the FP program SDp circuit 307, a program table formed of a plurality of FP 

included in the stream data transmitted from the data pro- programs transmitted from the data provider 302 side is 

vider 302 side is loaded (downloaded) into the FP circuit 307 stored in the memory 323. When the FP circuit 307 detects 

which performs the FP process. The FP circuit 307 uses the 25 a stream ID (or PID) of stream data SD from the data 

downloaded FP program SDp so as to perform the FP provider 302 side, the FP circuit 307 retrieves the FP 

process on the contents data transmitted from the data program corresponding to the stream ID from the program 

provider 302 side. Therefore, the FP circuit 307 can perform table 323 (steps S31 and S32). In the data receiving appa- 

the effective FP process which copes with the syntax of the ratus 301, a series of the processes after receiving the stream 

provided contents data. In other words, the data provider 302 30 data SD from the data provider 302 side is the same as that 

side can arbitrarily set the effecting FP process which copes according to the present embodiment. 

with the syntax, etc. of ' the provided contents data. When the FP circuit 307 cannot retrieve the FP program 

Therefore, it can flexibly cope with alternation of the FP f rom the program table 323, since an effective FP process 

process system and the CODEC system of contents data. As cannot be performed, the data reproduction/output process is 

a result, the data provider 302 side can securely protect the stopped (if NO at step S33, step S37 is performed). 

rights such as a copyright of multimedia provided by the FP The pp circxiit 307 dispatches corresponding FP program 

function without being limited by the CODEC system, etc. and p er f orms me effective FP process on the target contents 

of the contents data. data (step S34). Thereafter, in the similar manner, the 

j . An decoder 308 decodes (including expansion) the contents data 

Fourteenth Embodiment pD haye been &ubjcct (Q ^ pp prQcess and outputs 

FIG. 25 is a drawing showing a main section of the them into the interface 309 (step S35). The interface 309 

fourteenth embodiment. According to the present performs the signal converting process for converting into a 

embodiment, the decoder 308 of the thirteenth embodiment signal according to the specification of the data reproducing 

is divided into a syntax analysis circuit 380 A and a decoder 45 apparatus 310 connected to the data receiving apparatus 301 

380B in a narrow sense. The syntax analysis circuit 380A (step S3 6). 

performs the syntax analysis process on the contents data As described above, according to the present 

SDc separated and outputted by the data separating circuit embodiment, even if the stream data including plural types 

306, and analyzes a data structure, etc. of the contents data of contents data are provided simultaneously, if the FP 

SDc (a rule of a coded bit stream such as a hierarchical 5Q programs which cope with the respective types of contents 

structure, an order and position of data elements) and data are loaded by the data receiving apparatus 301, the 

outputs it into the FP circuit 307. effective FP process can be performed on the respective 

The FP circuit 307 uses the FP program SDp separated contents data. Moreover, the FP program can be provided in 

and outputted by the data separating circuit 306 so as to a packet different from the contents data. As an application 

perform the FP process for inserting specific data such as a 55 example of the present embodiment, a system, in which 

mark into a suitable position of the contents data SDc plural types of contents data are interleaved and provided, 

referring to the data structure, etc. outputted from the syntax and the FP process is performed in time division by the 

analysis circuit 380A. Therefore, the FP circuit 307 can respective FP programs, can be realized, 

insert specific data into a suitable position or an arbitrary This example is shown in FIG. 28. Plural contents data of 

position (excluded insert addition position) of the contents 50 plural types are formed into a packet and the packet is 

data SDc. transferred in an interleaved manner, such as TS packet of 

MPEG-2. The plural FP programs are respectively run in a 

Fifteenth Embodiment Ume divisional manner. 

FIG. 26 is a drawing showing a main section of the Contents data SDc is first stored in a buffer 402. Data for 

fifteenth embodiment. In the thirteenth embodiment, it is 65 identifying the contents in the packet (stream-id and/or PID 

assumed that the FP program SDp provided from the data in the case of MPEG-2) is read by an ID reader 404. The 

provider 302 side as data to be added to contents data SDc context controller 408 determines whether or not the present 
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packet belongs to the same contents stream which has been 
under the FP process at the time based on the contents 
identifying data. If the present packet belongs to the same 
contents stream which has been under the FP process at the 
time, the FP program which is now executed by a CPU 406 
is kept being executed in order to continue the FP process. 
If the present packet does not belong to the same contents 
stream which has been under the FP process at the time, the 
context controller 408 stops the FP program which is now 
executed by the CPU 406 and stores the context data (which 
is required to restart the FP process at later time) in the 
context store 410. It is determined whether or not the context 
store 410 stores the context data of the FP process for the 
contents of the new packet. If the context data is stored, the 
FP program corresponding to the contents of the packet and 
the context data corresponding to the contents of the packet 
are loaded into the CPU 406 and the FP program is executed 
in order to restart the FP program for the contents. If the 
context data is not stored, the FP program corresponding to 
the contents is only loaded into the CPU 406 and the FP 
program is executed in order to start the FP program for the 
contents. 

According to the present embodiment, plural types of 
contents data require the respective FP programs which cope 
with the respective type, but the FP process for the same type 
of plural contents data can be performed by one FP program. 
Therefore, in the case of receiving a plurality of contents 
data which are of the same type, the FP circuit 307 may 
perform the FP process using the common FP program. 

Even if the FP program corresponding to the same type 
(the CODEC system is the same) of contents data exists, 
when it is provided as different stream data, rightful owners 
of the contents data (creator and data provider) are not 
always the same. Therefore, the FP process according to 
different specific data or different algorithm should be per- 
formed on such contents data by different FP programs. 

Sixteenth Embodiment 

FIG. 29 is a block diagram showing a constitution of the 
data receiving apparatus 301 according to the sixteenth 
embodiment. The present embodiment is different from the 
thirteenth embodiment in that the data separating circuit 306 
includes a program separator 360 for separating a FP pro- 
gram SDp from stream data and a data separator 361 for 
separating video data SDv and audio data SDo from multi- 
plexed contents data. The FP circuit 307 has a function for 
performing the FP process on the video data SDv and audio 
data SDo separated by the data separator 361. The decoder 
308 includes a video decoder 362 for decoding (include 
expanding) the video data PDv which have been subjected 
to the FP process by the FP circuit 307, and an audio decoder 
363 for decoding the audio data PDo which have been 
subjected to the FP process. The interface 309 includes a 
video interface 364 for outputting video data from the video 
decoder 362 as a video signal, and an audio interface 365 for 
outputting audio data from the audio decoder 363 as an 
audio signal. 

According to such an arrangement, the data receiving 
apparatus 301 receives multiplex stream data of video and 
audio, and can perform the FP process on both the video and 
audio data. Therefore, even if the video or audio data are 
extracted from multimedia data received by the user side and 
recorded on DVD, for example, since the FP process can be 
performed on the respective data, the rights of the respective 
data can be protected. 

According to the present embodiment, the video decoder 
362 and the audio decoder 363 may be arranged so as to have 
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respective FP circuits. In this case, the data separating circuit 
306 separates FP programs corresponding to the video data 
SDV and the audio data SDo, and supplies the FP programs 
to the decoders 362 and 363 respectively. The video decoder 

5 362 performs the FP process on the video data SDv sepa- 
rated by the data separator 361 using the FP program 
supplied from the data separating circuit 306. Moreover, the 
audio decoder 363 performs the FP process on the audio data 
SDo separated by the data separator 361 using the FP 

1Q program supplied from the data separating circuit 361. 

As described above, according to the present invention, 
the data receiving apparatus comprises the first and second 
units, the first unit encrypts supplied data, which have been 
encrypted according to the first encrypt system, according to 

15 the second encrypt system so as to transfer them to the 
second unit, and the second unit decrypts the data according 
to the first encrypt system so as to returns them to the first 
unit. Therefore, since at least only the data which have been 
subject to the second encrypt appear in the interface between 

20 the first and second units, illegal use of data obtained by 
decrypting the encrypted data can be prevented. Moreover, 
since user's secrete data are not outputted to the outside of 
the second unit, the user's secret data can be protected 
securely. 

25 In addition, according to the present embodiment, ID data 
for effectively realizing the protection of rights of the data 
provider are inserted into provided data by the IC card 
prepared for each user. Therefore, where data reproduced by 
a user are copied into a recording medium and distributed 

30 illegally, the user who copies the data can be specified by 
reproducing ID data from the data. Further, since the ID data 
for protecting the rights are inserted by the I C card, it is not 
necessary to perform the data insert process by the data 
service provider side. As a result, it is possible to easily 

35 realize a data service system which provide data to many and 
unspecified users, such as a computer network and a digital 
broadcasting network. Moreover, ID data for effectively 
protecting the rights of the data provider can also be effi- 
ciently inserted into data reproduced by the data reproducing 

40 apparatus, which is connected to the data receiving appara- 
tus as local media. Further, in the data service system such 
as a digital broadcasting network, a user ID system for 
limiting a specific receiver is combined therewith, and thus 
it is possible to efficiently perform the ID data insert process. 

45 Furthermore, according to the present invention, in the 
user's data receiving apparatus side, it is possible to realize 
the FP process for inserting a mark for protecting the rights 
such as a copyright into a suitable position determined based 
on syntax data relating to a coding characteristic and data 

50 structure on stream data provided from the data provider 
side. Therefore, the FP process can be performed at real time 
without previously decoding the coded stream data. In other 
words, the process for compress-decoding which is neces- 
sary for the FP process of the mark insertion can be omitted. 

55 Moreover, since it is not necessary to perform the coding 
process on the stream data into which a mark has been 
inserted, a problem that the compression factor is lowered 
and a quantity of data is increased can be prevented. As a 
result, it is possible to realize the data providing service 

60 system which can protect the rights such as a copyright 
efficiently and securely. 

Further, according to the present invention, in the data 
receiving apparatus having the FP function, the FP function, 
which can cope with the CODEC system and FP process 

65 system adopted by the data providing service system, is 
realized, and thus the FP function, which can flexibly cope 
with old and new CODEC systems of the contents of 
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provided multimedia data and a new FP process system, can 8. A data receiving apparatus which is connected to a card 

be realized. Therefore, it is possible to securely protect the module having data inserting means for inserting predeter- 

rights such as a copyright of the multimedia data in the data mined ID data into input data transmitted from an external 

providing service and thus to construct the data providing apparatus and which receives the input data, said data 

service system which made efficient distribution possible. 5 receiving apparatus comprising: 

Additional advantages and modifications will readily means for receiving the input data transmitted from the 

occur to those skilled in the art. Therefore, the present external apparatus; 

invention in its broader aspects is not limited to the specific interface means connected to said card module; and 

details, representative devices, and illustrated examples control means for transmitting the input data received by 

shown and described herein. Accordingly, various modifi- 1Q sa fd receiving means to said card module connected to 

cations may be made without departing from the spirit or sa id interface means and making the ID data inserted 

scope of the general inventive concept as defined by the into the input data by using said data inserting means so 

appended claims and their equivalents. For example, each as to receive the input data into which the ID data have 

embodiment describes independently, but a plurality of Deen inserted from said card module, 

embodiments may be suitably combined. The FP process The apparatus according to claim 8, in which said data 

may be combined with the first embodiment. In this case, the 1 inserting means concealedly inserts the ID data, which 

function of the IC card 110 (data insert unit 120) in the identifies a user who is provided with data from a data 

second through ninth embodiments, or the FP circuit 206 in creator or provider and which can be reproduced by a 

the eleventh and twelfth embodiments, or the FP circuit 307 predetermined reproducing process, into the input data 

in the thirteenth through sixteenth embodiments may be according to a predetermined process, 

provided to the security module 70 in the first embodiment 20 The apparatus according to claim 8, in which said data 

(FIG. 2). inserting means concealedly inserts the ID data into the 

What is claimed is: input data according to a process based on a program for 

1. A data receiving apparatus comprising: inserting the ID data included in the input data transmitted 
a first unit for receiving encrypted data that have been from the external apparatus. 

encrypted according to a first system; and 25 11. The apparatus according to claim 9 or 10, in which the 

a second unit connected to said first unit detachably, process which is performed by said data inserting means is 

wherein said first unit comprises means for encrypting, a data hiding process or a watermarking process. 

according to a second system, the received data, which 12. The apparatus according to claim 8, in which said 

have been encrypted according to the first system, receiving means comprises an interface connected to a 

means for outputting the doubly encrypted data to said 30 computer network, communication channel or broadcasting 

second unit, and means for decrypting data supplied network. 

from said second unit according to the second system, 13. The apparatus according to claim 8, in which said 

anc j receiving means comprises an interface connected to a data 

said second unit comprises means for decrypting the recording medium. 

doubly encrypted data supplied from said first unit 3S 14. The apparatus according to claim 8, further compris- 

according to the first system and means for transmitting ing reproducing means for reproducing the input data, into 

the data back to said first unit. which the ID data have been inserted and which are out- 

2. The apparatus according to claim 1, in which said first putted from said control means using a predetermined 
unit comprises key generating means for generating an reproducing apparatus. 

encrypt/decrypt key of the second system, and the key 40 15. A card module adapted to be connected to a data 

generated from said key generating means is not outputted receiving apparatus, comprising: 

to the outside of said first unit. a memory for storing the ID data; and 

3. The apparatus according to claim 2, in which the data inserting means for receiving data received by said 
encrypt/decrypt key of the second system generated from data receiving apparatus and inserting the ID data into 
said key generating means of said first unit is variable. 45 received data and for transmitting the received data 

4. The apparatus according to claim 1, in which said back to said data receiving apparatus. 

second unit comprises a memory for storing a decrypt key of 16. The card module according to claim 15, in which: 

the first system thereinto, and the key read out from said sa jd data inserting means comprises descrambling means 

memory is not outputted to the outside of said second unit. f or receiving scrambled data from said data receiving 

5. The apparatus according to claim 1, in which said 50 apparatus and descrambling received data, and 
second unit comprises means for inserting ID data peculiar sa j d data i nser ting means inserts the ID data into an output 
to said second unit into the data supplied from said first unit. f rom descrambling means. 

6. The apparatus according to claim 5, in which said 17 card mo dule according to claim 15, in which said 
inserting means comprises: da t a inserting means concealedly inserts the ID data, which 

syntax analysis means for analyzing syntax of the data 55 identifies a user who is provided with data from a data 

supplied from said first unit, and obtaining syntax data creator or provider and which can be reproduced by a 

representing a coding characteristic and data structure predetermined reproducing process, into the received data 

of the data supplied from said first unit; according to a predetermined process. 

means for determining an insert position according to the 18. The card module according to claim 15, in which said 

syntax data; and 60 data inserting means concealedly inserts the ID data into the 

means for inserting the ID data into the position of the received data according to a process based on a program for 

data supplied from said first unit determined by said inserting the ID data included in the received data transmit- 

determining means. ted from the external apparatus. 

7. The apparatus according to claim 5, in which said 19. The card module according to claim 17 or 18, in which 
inserting means inserts the ID data based on an insert 65 the predetermined process which is performed by said data 
process program included in the data supplied from said first inserting means is a data hiding process or a watermarking 
unit. process. 
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20. A data receiving apparatus which is connected to a 
card module having authentication means for authenticating 
a user who is provided with data from a data creator or 
provider and which receives input data transmitted from an 
external apparatus, said apparatus comprising: 

means for receiving the input data; 

interface means connected to a card module; 

data inserting means for making card module connected 
to said interface means to perform a user authentication 
process on the data received by said receiving means, 
inputting ID data which identifies the user and which is 
obtained by the authentication process, and for insert- 
ing the ID data which can be reproduced into the input 
data received by said receiving means; and 

control means for outputting data including the ID data 
which have been inserted by said data inserting means. 

21. The apparatus according to claim 20, in which said 
data inserting means concealedly inserts the ID data into the 
input data according to a process based on a program for 
inserting the ID data included in the input data transmitted 
from the external apparatus. 

22. A data receiving apparatus for receiving data trans- 
mitted from an external apparatus and outputting the data 
into a reproducing apparatus, comprising: 

data inserting means for inserting ID data which can be 
reproduced into received data; and 

output means for outputting the received data, into which 
the ID data have been inserted by said data inserting 
means, into said reproducing apparatus. 

23. A data receiving apparatus for receiving a stream data, 
which is transmitted from an external apparatus and formed 
of a bit stream which has been coded by a predetermined 
coding system, and performing an outputting process 
according to a specification of a reproducing apparatus, said 
data receiving apparatus comprising: 

means for receiving the stream data; 

syntax analysis means for performing a syntax analysis 
process on the stream data received by said receiving 
means and obtaining syntax data relating to a coding 
characteristic and data structure of the stream data; 

determining means for determining an insert position in 
the stream data, into which specific data which can be 
reproduced are concealedly inserted, based on the syn- 
tax data obtained by said syntax analysis means and 
predetermined parameter data; 

inserting means for inserting the specific data into the 
insert position in the stream data determined by said 
determining means; and 

output means for performing a predetermined output 
process according to a predetermined specification of 
reproduction/output on the stream data outputted from 
said inserting means. 

24. The apparatus according to claim 23, in which said 
output means comprises decoding means for performing a 
predetermined decoding process on the stream data output- 
ted from said inserting means. 

25. The apparatus according to claim 23, in which: 

the parameter data is data which includes user specific 
data for specifying a user, use terminal specific data for 
specifying a use terminal apparatus used by a user, use 
time data and use area specific data and which sets the 
specific data, and 

said determining means determines an insert position in 
the stream data where a bad influence, which is exerted 
on reproduction/output characteristic and reproducibil- 
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ity of the specific data upon reproducing/outputting 
contents data included in the stream data, can be 
suppressed when the specific data set according to the 
parameter data are inserted. 

26. The apparatus according to claim 23, which further 
comprises: 

memory means for storing data of the insert position 
determined by said determining means as insert posi- 
tion history data, and in which 
said determining means uses the insert position history 
data stored in said memory means when determining 
an insert position in a new stream data of the specific 
data. 

27. The apparatus according to claim 23, in which said 
inserting means performs a program for executing a data 
hiding process for concealedly inserting prepared specific 
data which can be reproduced and identified, a watermarking 
process or a fingerprint process so as to insert the specific 
data into the insert position in the stream data determined by 
said determining means. 

28. The apparatus according to claim 23, in which: 

the data transmitted from the external apparatus has 
contents data which can be used by an image repro- 
ducing apparatus and an audio reproducing apparatus 
and which have been coded by a predetermined mul- 
timedia coding system, and is a stream data, which is 
transmitted via a computer network, a digital commu- 
nication channel or a digital broadcasting network, 
according to a predetermined synchronizing specifica- 
tion and multiplexing specification, and 

said output means comprises means for decoding the 
contents data outputted from said data inserting means 
based on the multimedia coding system, and for per- 
forming a signal process, which is required for the 
image reproducing apparatus or audio reproducing 
apparatus, on the decoded contents data. 

29. The apparatus according to claim 23, which further 
comprises: 

conversion table generating means for generating conver- 
sion table data for referring to a conversion result of the 
specific data in the partial stream constitution corre- 
sponding to the insert position in the stream data 
determined by said determining means; and 

conversion table storing means for storing the conversion 
table data, and in which 

said inserting means performs the data insert process 
for inserting the specific data based on the insert 
position in the stream data determined by said deter- 
mining means and the conversion table data in said 
conversion table storing means. 

30. A data inserting method, which is applied to a data 
receiving apparatus having functions for receiving data to be 
transmitted as a stream data formed of a bit stream, which 
has been coded by a predetermined coding system, from the 
outside, for outputting the data according to a predetermined 
specification of reproduction/output, and concealedly insert- 
ing specific data which can be reproduced and identified into 
the stream data, said method comprising the following steps 
of: 

performing a syntax analysis process on the received 
stream data and obtaining syntax data relating to a 
coding characteristic and data structure of the stream 
data; 

determining an insert position in the stream data into 
which the specific data which can be reproduced are 
concealedly inserted based on the syntax data obtained 
by said obtaining step and predetermined parameter 
data; 
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inserting the specific data into the insert position in the 
stream data determined by said determining step; and 

outputting the stream data, into which the specific data 
have been inserted by said inserting step, according to 
the predetermined specification of reproduction/output. 

31. A data receiving apparatus comprising: 
means for receiving data transmitted from an external 

apparatus; 

extracting means for extracting a program which is 
included in the data received by said receiving means 
and which is used for concealedly inserting specific 
data which can be reproduced into specified data; and 

data inserting means for inserting the specific data into the 
specified data included in the data received by said 15 
receiving means using the program extracted by said 
extracting means and outputting the specified data into 
which the specific data is inserted. 

32. A data receiving apparatus comprising: 
means for receiving data transmitted from an external 20 

apparatus; 

extracting means for extracting a program which is 
included in the data received by said receiving means 
and which is used for concealedly inserting specific 
data which can be reproduced into specified data; 25 

data inserting means for inserting the specific data into the 
specified data included in the data received by said 
receiving means using the program extracted by said 
extracting means and outputting the specified data; and 

output means for outputting the data outputted from said 
data inserting means. 

33. The apparatus according to claim 31 or 32, in which: 
the data transmitted from the outside includes contents 

data which can be used by reproduction/output and the 
program, and 

said extracting means comprises means for separating the 
program corresponding to the contents data from the 
data received by said receiving means based on ID data 
for identifying the contents data and outputting the 40 
program. 

34. The apparatus according to claim 31 or 32, in which 
said data inserting means comprises: 

memory means for loading the program extracted by said 
extracting means; and 

means for performing a data hiding process, watermark- 
ing process or fingerprint process for concealedly 
inserting the specific data which can be reproduced into 
the contents data included in the data received by said 
receiving means based on algorithm of the program. 

35. The apparatus according to claim 31 or 32, in which 
said data inserting means comprises: 

memory means for storing plural types of programs 
transmitted from the outside; and 

means for inserting the specific data by retrieving the 
program corresponding to the contents data from said 
memory means and by using the program based on ID 
data for identifying the contents data included in the 
data received by said receiving apparatus. 



36. The apparatus according to claim 32, in which: 
the data transmitted from the outside includes contents 

data which can be used by video output and audio 
output and which has been coded by a predetermined 
multimedia coding system, and 
said output means comprises means for decoding the 
contents data outputted from said data inserting means 
based on the multimedia coding system and for per- 
forming a signal process, which is required for the 
video output and audio output, on the decoded contents 
data. 

37. The apparatus according to claim 31 or 32, in which: 

the data transmitted from the outside is a stream data 
according to predetermined synchronizing and multi- 
plexing specifications transmitted via a computer 
network, digital communication channel or digital 
broadcasting network, and 
said receiving means comprises an interface which is 
connected to the computer network, digital communi- 
cation channel or digital broadcasting network. 

38. A data inserting method which is applied to a data 
receiving apparatus having functions for receiving data 
transmitted from an external apparatus, outputting contents 
data, which can be used and included in the data, according 
to reproduction/output forms, and concealedly inserting 
specific data which can be reproduced and identified into the 
contents data, said method comprising the following steps 
of: 

extracting a program used for performing the insert pro- 
cess from the data transmitted from the external appa- 
ratus; 

storing the program extracted by said extracting step into 
memory means according to ID data for identifying the 
contents data; and 
concealedly inserting the specific data into the contents 
data by using the program stored in the memory means 
by said storing step. 

39. A data inserting method which is applied to a data 
receiving apparatus having functions for receiving data 
transmitted from an external apparatus, outputting contents 
data, which can be used and included in the data, according 
to reproduction/output forms, and concealedly inserting 

45 specific data which can be reproduced and identified into the 
contents data, said method comprising the following steps 
of: 

extracting a program used for performing the insert pro- 
cess from the data transmitted from the outside; 
storing plural types of the programs extracted by said 
extracting step into memory means according to ID 
data for specifying the contents data; 
retrieving a program corresponding to the contents data 

from said memory means based on the ID data; and 
concealedly inserting the specific data into the contents 
data by using the program retrieved from the memory 
means by said retrieving step. 
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